Papers with world model
PLM-based World Models for Text-based Games (2022.emnlp-main)
Copied to clipboard
| Challenge: | a new study shows that pre-trained world models provide a strong base for world models . worldformer is a text-based game environment that can be used to learn world models in text-driven games. |
| Approach: | They propose to use pre-trained language models to build world models in text-based game environments. |
| Outcome: | The proposed model outperforms state-of-the-art model-free algorithms in Atari games while retaining sample efficiency. |
Gaussian Process based Deep Dyna-Q approach for Dialogue Policy Learning (2021.findings-acl)
Copied to clipboard
| Challenge: | Reinforcement learning (RL) is the main dialogue policy learning method in recent years. |
| Approach: | They propose a Gaussian Process based Deep Dyna-Q approach to dialogue policy learning . they propose evaluating the quality of experiences generated by the world model using a discriminator . |
| Outcome: | The proposed approach improves the effectiveness and efficiency of dialogue policy learning by 20% with fewer human-machine interactions. |
Deep Dyna-Q: Integrating Planning for Task-Completion Dialogue Policy Learning (P18-1)
Copied to clipboard
| Challenge: | Training a task-completion dialogue agent via reinforcement learning (RL) is costly because it requires many interactions with real users. |
| Approach: | They propose a framework that integrates planning for task-completion dialogue policy learning into a dialogue agent using a world model to mimic real user response and generate simulated experience. |
| Outcome: | The proposed framework integrates planning for task-completion dialogue policy learning with real user interaction and simulated user behavior. |
An Efficient Dialogue Policy Agent with Model-Based Causal Reinforcement Learning (2025.coling-main)
Copied to clipboard
| Challenge: | Existing models for dialogue policy training consider one-step dialogues, leading to inaccurate simulations. |
| Approach: | They propose a framework for dialogue policy learning that trains an agent to select dialogue actions via deep reinforcement learning. |
| Outcome: | The proposed framework achieves state-of-the-art performance on three dialogue datasets . it uses model-based reinforcement learning with automatically constructed causal chains . |
Making Large Language Models into World Models with Precondition and Effect Knowledge (2025.coling-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are not inherently designed to model real-world dynamics, but can be induced to perform two critical world model functions: determining the applicability of an action based on a given world state and predicting the resulting world state upon action execution. |
| Approach: | They propose to use Large Language Models to model world states and preconditions . they validate that precondition and effect knowledge generated by LLMs aligns with human understanding of world dynamics . |
| Outcome: | The proposed model can predict valid actions and state transitions, thereby replicating existing models. |
Efficient Integration of External Knowledge to LLM-based World Models via Retrieval-Augmented Generation and Reinforcement Learning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing attempts to enhance LLM-based world models through prompting or fine-tuning approaches are either requiring human knowledge or computationally extensive. |
| Approach: | They propose a framework that leverages retrieval-augmented generation to integrate external knowledge to LLM-based world models. |
| Outcome: | The proposed framework outperforms baseline models and exhibits strong generalizability. |
Reasoning with Language Model is Planning with World Model (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown remarkable reasoning capabilities, particularly with Chain-of-Thought-style prompts. |
| Approach: | They propose a framework that repurposes the LLM as both a world model and a reasoning agent and incorporates a principled planning algorithm (based on Monte Carlo Tree Search) |
| Outcome: | The proposed framework repurposes the LLM as both a world model and a reasoning agent and incorporates a principled planning algorithm (based on Monte Carlo Tree Search) it achieves optimum balance between exploration and exploitation, while achieving high-reward reasoning paths efficiently. |
Model-Based Simulation for Optimising Smart Reply (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods to learn to predict responses to messages are based on post-hoc diversification rather than learning to predict sets of responses. |
| Approach: | They propose a method that employs model-based simulation to discover high-value response sets by simulating possible user responses with a learned world model. |
| Outcome: | Empirically, the proposed method improves ROUGE score and Self-ROUGE scores on two public datasets compared to baselines. |
CoEx – Co-evolving World-model and Exploration (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing LLM agents fail to assimilate new observations into dynamic updates of the world model, leading to divergent and erroneous plans. |
| Approach: | They propose a hierarchical agent architecture that allows LLM planning to co-evolve with a dynamically updated model of the world. |
| Outcome: | The proposed agent outperforms existing agent paradigms in planning and exploration. |
Deliberate Reasoning in Language Models as Structure-Aware Planning with an Accurate World Model (2025.acl-long)
Copied to clipboard
| Challenge: | Existing Chain-of-Thought (CoT) methods struggle with consistency and verification in complex reasoning tasks. |
| Approach: | They propose a framework that integrates structured knowledge representation with learned planning. |
| Outcome: | The proposed framework outperforms existing Chain-of-Thought (CoT) methods on math reasoning, logical reasoning, and coding tasks. |